System and method for 3D projection mapping with robotically controlled objects

ABSTRACT

A system for motion control is presented. In one embodiment, a motion control 3D projection system includes a projector; and a projection surface coupled to a robotic arm, where the robotic arm moves the projection surface through a set of spatial coordinates, and a 3D projection from the projector is projected onto a set of coordinates of the projection surface and matches the 3D projection to the set of coordinates of the projection surface as the projection surface moves through the set of spatial coordinates. In additional embodiments, a master control system may integrate additional robotic arms and other devices to create a motion control scene with a master timeline.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a non-provisional application of U.S. ProvisionalPatent Application No. 61/560,744, filed Nov. 16, 2011, entitled “Systemand Method for 3D Projection Mapping with Robotically ControlledObjects”, and is incorporated by reference herein in its entirety forany and all purposes.

BACKGROUND

Embodiments of the present invention relate to the use of automatedsystems and controls in the creation of audiovisual presentations. Morespecifically, embodiments of the invention relate to systems and methodsof using a projection mapping with robotic motion control.

3D projection mapping is a technique to create the illusion of objectsand environments where they don't physically exist. By projecting lighton a surface that is rendered from a specific perspective the viewerperceives an environment that is not actually physically present.

Motion control is a type of automation, where the position of an objector objects is controlled using some type of device such as a roboticarm. In the production of videos, films, commercials, movies, and othersuch audiovisual works, the placement and movement of the camera andobjects in a scene being captured by the camera is a major considerationand source of time and expense. The use of motion control is known inthe film industry, where a camera is mounted and controlled as part ofthe creation and recording of a video scene. This technique is commonlyreferred to as motion control photography.

SUMMARY

Various embodiments of motion control systems are presented. In onepotential embodiment, a 3D projection surface is matched with aprojector. Either or both items may be mounted on robotic arms whichfunction as device actors for a scene that is set to vary over time andspace, with the device actor moving the projection surface and theprojector presenting a projected image onto the projection surface. Inone embodiment, this enable motion control of a scene actor that ispartially mechanically animated using the device actor and partiallycomputer animated using the projection from the projector.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram of a motion control photography systemincluding a master scene control according to one embodiment of theinnovations herein.

FIG. 2 shows a perspective view of a robot for use with a motion controlphotography system according to one embodiment of the innovationsherein.

FIG. 3a shows a view of a robot with 7 degrees of freedom according toone embodiment of the innovations herein.

FIG. 3b shows a view of a robot with an attached camera according to oneembodiment of the innovations herein.

FIG. 3c shows a view of a robot with an attached lighting unit accordingto one embodiment of the innovations herein.

FIG. 4a shows a view of wrist mount for use with a robotic arm inaccordance with one embodiment of the innovations herein.

FIG. 4b shows a view of a robot with an attached wrist mount accordingto one embodiment of the innovations herein.

FIG. 5a shows one potential embodiment of a master input for use with amotion control system according to one embodiment of the innovationsherein.

FIG. 5b illustrates a global timeline coordinating device actorpositions through a master control according to one embodiment of theinnovations herein.

FIG. 6a shows a block diagram of a motion control photography systemincluding a master scene control according to one embodiment of theinnovations herein.

FIG. 6b illustrates a method of control according to one embodiment ofthe innovations presented herein.

FIG. 7 shows one potential implementation of a computer or electronicdevice in accordance with various embodiments of the presentinnovations.

FIG. 8 illustrates a potential user interface for a software controlaccording to one embodiment of the innovations herein.

FIG. 9 illustrates a potential user interface for a software controlaccording to one embodiment of the innovations herein.

FIG. 10 shows a block diagram of a motion control photography systemincluding a master scene control according to one embodiment of theinnovations herein.

FIG. 11 shows one potential implementation of a system including 3Dprojection mapping.

FIG. 12 shows one potential implementation of a projector in a systemincluding 3D projection mapping.

FIG. 13 shows one potential implementation of a system including 3Dprojection mapping.

FIG. 14 shows one potential implementation of a system including 3Dprojection mapping.

FIG. 15 shows one potential implementation of a system including 3Dprojection mapping.

FIG. 16 shows one potential implementation of a system including 3Dprojection mapping.

FIG. 17 shows one potential implementation of a system including 3Dprojection mapping.

FIG. 18 shows one potential implementation of a system including 3Dprojection mapping.

FIG. 19 shows one potential implementation of a system including 3Dprojection mapping.

FIG. 20 shows one potential implementation of a user interface for asystem including 3D projection mapping.

FIG. 21 shows one potential implementation of a user interface for asystem including 3D projection mapping.

DETAILED DESCRIPTION

Embodiments of the innovations disclosed herein include systems andmethods for providing enhanced motion control with three dimensional(3D) projection mapping. In particular, systems and methods describe asystem with a three dimensional projection surface attached to acontrollable robotic arm, a projector for projecting an animatedprojection onto the 3D projection surface, and an attached system forsynchronizing movement of the 3D projection with projection from theprojector onto the 3D projection surface.

One potential embodiment is used in the field of animatronics, orcreating animated robots that appear lifelike. In such an embodiment, aprojection surface is shaped in the form of a represented creature suchas a dinosaur, a monkey, or some other creature. Certain portions of thecreature is difficult to create using physical mechanical devices Facialfeatures such as eyes and lips, in particular, have historically bedifficult to recreate mechanically. One or more projectors is positionedwith an optical path from the projector to all or part of the projectionsurface, and an animated projection of eyes and a mouth is projectedonto the projection surface. The projection surface is then be moved toimitate walking, head movement, or other characteristic movement of thecreature being simulated. As this movement of the projections surfaceoccurs, the projection of eyes and a mouth matches the movement of theprojection surface, so the position of the eyes and mouth remainrelatively stationary on the projection surface, while the illusions ofblinking eyes and mouth moment occurs. If such a system includessufficiently powerful computer animation software, the motion of theprojected features is controlled real-time to, for example, match mouthmovements to sounds being created which enables non-scriptedconversation with the animatronic robot.

Further, some embodiments of such a projection system include multipleprojection surfaces, multiple projectors, and additional system devicessuch as sound, ambient lighting, and other such effects that isintegrated into a master control system to enable forward and reversemotion through a series of scripted and synchronized sets and acontrollable rate. Additional embodiments further include integratedsafety features for preventing collisions between devices or personsoperating within an area where the devices are moving.

FIG. 1 describes a motion control system 100 Motion control system 100is part of a motion controlled set used to film a scene using motioncontrol photography. Motion control system comprises a master control10, input and feedback systems 20, device actors 40, and safety systems90. From the most basic perspective, motion control system 100 functionswhen an input system 20 provides instructions to a device actor 40 viamaster control 10.

For the purposes of the present invention, a scene comprises a set ofmotions and actions by device actors 40 over a continuous period oftime, such that a set of players in front of a camera is recorded invideo, sound, or both. The players are people, stationary objects, orobjects controlled or moved by one or more devices of device actors 40.In one embodiment, the camera is mounted to a robot arm of device actors40. At a beginning of a scene, a camera and a plurality of players beginin a first position. Motion control of device actors 40 moves the cameraand the players through a sequence of motions to the end of a scene,with players and sound from the scene recorded using the camera andpotentially other audio and video recording equipment to capture themotion.

In one potential embodiment as part of a motion control system 100,input and feedback systems 20 include a database 22, a master input 24,a software control 26, and an independent manual control 28. As part ofthe input and feedback systems 20, database 22 operates to provide a setof timing and position data to direct all or a portion of device actors40. Alternatively, database 22 stores data being created by manual orindividual movement or data input related to operation and function ofdevice actors 40. Database 22 also stores data created independently ofdevice actors 40, such as data created using software modeling featuresof a software control 26.

A master input 24 is any device that functions to operate all of thedevice actors 40 associated with a particular scene being created withmotion control system 100. Master input 24 functions by sending inputcontrol signals to master control 10. Master control 10 then adapts thesignal from master input 24 to send individual control signals to aplurality of actors operating as device actors 40 for the particularscene. In one potential embodiment, every individual device of deviceactors 40 is provided a control signal from master control 10 when asignal is received from master input 24, including a signal to maintaina status quo or non-action to devices that are not operating as deviceactors 40 for a particular scene. In an alternative embodiment, aportion of the device actors connected as part of motion control system100 are not sent any signal from master control 10 as part of theoperation of motion control system 100 for a particular scene.

Software control 26 acts as a replacement for master input 24 in sendingcontrol signals to the plurality of actors via the master control 10.Alternately, software control 26 controls individual devices from amongdevice actors 40 to alternate, change, or experiment with motions of theindividual device. In other potential embodiments, software control 26functions to model the behavior of individual devices of device actors40 within a virtual environment. In such an embodiment, software control26 contains a software model for an individual device, which allowscontrol signals to be created for the device without actually sendingthe control signals to the device. The control signals are then bestored in the software control 26, in database 22, within a computermemory component that is part of master control 10, or within computermemory that is part of the device of device actors 40 for which thecontrols are being created. After the control signal is created bysoftware control 26 and propagated to the appropriate storage location,a master control signal from software control 26 or from master input 24activates the control signal for the individual device to act inconjunction with other device actors 40.

Certain devices of device actors 40 additionally have an independentmanual control 28. As described above with respect to software control26, control signals for an individual device are created in softwaremodeling. Similarly, a device may have independent manual control 28that is used to operate a device of device actors 40. For example, inone potential embodiment, a device that is one of device actors 40 is afollow focus that controls the focus of a camera. The follow focus mayhave a control device that is designed to control the focus of thecamera that may operate as an independent manual control. When a set ofinstructions is being created for the entire scene, the independentmanual control 28 is given input commands over time that are recorded todatabase 22 or a memory device of master control 10. During creation ofa set of instructions using independent manual control 28, theindependent manual control 28 may communicate directly with theassociated device of device actors 40. Alternatively, the independentmanual control 28 may send the control signal to master control 10,which then conveys the signal to the associated device of device actors40. The control signal may then be created either from the signal of theindependent manual control 28, of from a measured feedback readingcreated by the operation of the associated device. Additionally,although in many situations it is preferable to have the independentmanual control 28 actually control the associated device during controlsignal creation in order to view the results, control signals is createdwithout controlling the device. For example, if expected input signalsare expected for certain time marks, an independent manual control 28 isoperated independent of the related device, and the control operationrecorded. These function as ways in which instructions for individualdevice actors of device actors 40 is integrated into a motion controlledscene as part of motion control system 100.

The result of the integration described above is considered as a globaltimeline for a motion controlled scene. FIG. 5b , described in moredetail below, provides an illustrated example, where the actions ofmultiple motion controlled actors are integrated into a global timelineusing a master control. In FIG. 5b , various actors such as camera andlighting robots move at certain points during a scene. The actors isreceiving control signals during the entire scene from time a to time f,or may only receive controls when they actually move or act. Certainother actors, such as 1^(st) and 2^(nd) special effects (fx) actors onlyreceive single commands to act at one specific time, such as 1^(st) fxactor acting at time b in the global timeline of FIG. 5b . Such anintegration into a global timeline allows simplified user control ofcomplex actions impacting an entire scene that may save significantamounts of time in modifying a particular scene with a given timeline.This allows scrubbing through time forward and backwards as well asseeking to specific frame number or timecode for all actors in aparticular scene, and slowing down or speeding up the performance of theentire set (system) in real time via hardware device

Although embodiments will be illustrated with reference to specificimplementations, a person of ordinary skill in the art will understandthat these implementations describe innovations which may have broad useother than the specifically described implementations. As describedbelow, enhanced control may comprise systems and methods for a varietyof functions, including safety systems, playback speed control, forwardand reverse position scrubbing, and integrated detection systems, amongothers.

Such a system includes advantages over systems currently known in theart by providing accessible and highly sophisticated robotic controls inan art environment dominated by custom toolsets without simple ormalleable integration toolsets. Such a use of highly accurate controlsystems with cameras is considered, in certain embodiments, to be“cinematic automation” or “3D projection automation” which allows theambitions of visual story tellers to be matched through the applicationof automation systems. For example, improved control systems cancoordinate sub-millimeter position of a camera in space with theposition of the light, an actress, a robotic actor comprising a 3Dprojection surface, and special effects (pyrotechnics, video playback,sound cues, etc.). This allows execution of highly complex shots thatwould previously have required the coordination of several filmdepartments with manual human positioning. Such control systems removeinaccuracy and introduced the repeatability of robotics, through highlyaccurate computer synchronization of events. In addition to developing afaster, more rigid, safer, and easier to program robotic arm system,embodiments of the present innovations include interfaces that allow acreative director to make very quick on-set adjustments. In the highpressure environment of feature films and commercial productions, it iscritical that a director or visual effects supervisor is able to makevery quick creative or technical calls, and the systems described hereinenable this in a way not known in the previous art.

As a further example, if a system implementation of the innovations isbeing used in presentation of a scene in a live play, the system mightbe synchronizing a large robotic arm, a custom robotic rig to move thearm, video playback of a wall of LED's that serve as the light source,and the motion of a piece of background, and an animatronic actor withfacial features animated from a projector attached to an animationcomputer with arm and leg movement animated by robotics. This is ahighly technical, pre-programmed scene that has been pre-visualized in acomputer, and the interplay of all the elements have been choreographedusing the system. Real time during the play, if the actors aredelivering their lines at a slower than normal rate, or there is a needto pause for extra applause, a play director may compensate real time bypausing the entire set or by simply turning a knob at the right momentto accommodate the actors. The robot slows down, the robotic rigcomplies, and the entire scene of the play decelerates. All of thishappens in synchronicity. Plus the system can provide integratedenhanced safety to prevent the actors from being injured.

Referring now to FIGS. 2-4, several non-limiting examples of deviceactors 40 will be described. Although these figures focus on the use ofrobotic arms, device actors is other types of devices that isstationary, such as sensors, stationary lights, and signal sources, aswill be described later.

FIG. 2 describes device actor 242. Device actor 242 comprises roboticarm 246, mounting point 248, and rail 244. Device actor 242 is describedin FIG. 2 with no specific functionality attached at any mounting pointsuch as mounting point 248. Mounting point 248 is configured to hold acamera, a light, a player that will be filmed by a camera, or any otherrelevant device or object. In certain embodiments, instructions is givento position mounting point 248 at a specific location, and the positionsof the axis of robotic arm 246 and of rail 244 is calculated by aprocess of the related motion control system. In alternativeembodiments, each axis of robotic arm 246 and the position of rail 244require separate, individual settings and control commands.

FIG. 3a describes device actor 342, comprising a robotic arm with axisA1-A6, with axis A0 associated with a rail which is not shown thatallows side to side movement of the other eight axes. FIG. 3b describesdevice actor 342 with a camera mount 352 placed at a mounting point, andcamera 350 attached to camera mount 352. FIG. 3c describes device actor342 with light 360 placed at a mounting point.

FIGS. 4a and 4b describe an embodiment where a device actor 442comprises a robotic arm 444 with a wrist mount 410 and a wrist mountinterface 412. Wrist mount 410 and wrist mount interface 412 enablesmultiple device actors to be mounted to robotic arm 444 in addition tostandard mounts such as a camera or lighting fixture. In certainembodiments, the wrist mount interface enables temperature sensors,laser range detectors, microphones, speakers, fans, or other mechanicalactivated or special effects devices.

FIG. 5a describes master input 524 which is one potential implementationof a master input such as master input 24 of FIG. 1. Master input 524comprises a display interface 510, and engage control 540, a mode select530, and an analog interface 520. The function of master input 524 willbe described in conjunction with the global timeline of FIG. 5 b.

FIG. 5b illustrates a global timeline associated with one potentialembodiment of a motion controlled scene implemented in a motion controlsystem such as motion control system 100 of FIG. 1. FIG. 5b includes anumber of actors, as defined above, which operate during a scene with anominal or standard running time from time a to time f. The globaltimeline is nominal and not referring to real clock time because, aswill be detailed further below, master input controls alters the rate atwhich actors progress through motions from the beginning of a scene tothe end of a scene. The global timeline may therefore alternately bethought of as a coordinated position chart for actors. During the scenerunning time, an actor may act or move during the entire scene, during aportion of the scene, or only for one instant of a scene. Each actor,though, will have a predetermined point for each nominal time from thebeginning to the end of the scene. The predetermined points for eachactor are associated with predetermined points for each other actor.

In one potential embodiment, individual control signals for specificdevice actors are coordinated into a single file within a memory of amaster control with a common base time provided by a master clock withinthe master control. During operation, master control extracts controlsignals for each device actor and provides individual control signals toeach device actor at the appropriate intervals. In an alternativeembodiment, a master control maintains separate individual controlsignal files and timing data for different device actors, andsynchronizes the different control signals separately from theindividual control files.

In another alternative embodiment, the control data for a portion of thedevice actors is transferred by a master control to a memory within anassociated individual device actor. During operation, device actorshaving control data within memory receive only a synchronization signalthat indicates a location in a global timeline, a rate of progressthrough a global timeline, or both.

The specific embodiment described in FIG. 5b includes a camera robot, afollow focus, two lighting robots, a player robot that controls anobject that is to appear in front of a camera, and two special effectsactors. As described above, each actor has a path to follow through thescene from beginning at time a to end at time f. In the specificembodiment of FIG. 5b , each actor begins at a preset position. Asdescribed by the global timeline, only the player robot moves from timea to time b. This will be true whether b occurs 10 seconds afterstarting time a, or 20 seconds after time a due to modification of therate at which the scene progresses by a master input, as is furtherdetailed below. At time b, 1^(st) special effects actor is activatedwith a single command, player robot continues moving, and follow focusand camera robot begin moving.

The chart of FIG. 5b is not meant to indicate that non-moving actorssuch as 1^(st) lighting robot, 2^(nd) lighting robot, and 2^(nd) fxactor during time a to time b are not receiving input commands. FIG. 5bmerely illustrates a global position timeline. In certain embodimentsthey may not be receiving input commands from a master control. Inalternative embodiments, however, non-moving actors such as 1^(st)lighting robot may periodically or continuously receive commands from amaster control, even when not moving, where the command is aninstruction not to move. Such a command may function to maintainsynchronization, acting as a clock or a timing heartbeat to maintainsynchronization. In some embodiments, real time adjustments are made toan actor during a scene. Such a clock or timing heartbeat further servesto provide synchronization data as the adjustments feed back to themaster control to update and change the global timeline.

Referring back to FIG. 5a , master input 524 includes a displayinterface 510. Display interface 510 is a modular touch screen devicethat displays status information related to actors or the globaltimeline. For example, display interface 510 includes a scene timeassociated with the current position of the actors in the globaltimeline, or a frame number for a current scene associated with thecurrent timeline. In such a display, time a, for example, or frame 1, isdisplayed when the actors are in position for the beginning of thescene. In one potential embodiment, display interface 510 is a portableelectronic device or a cellular telephone with an interface applicationthat communicates with a master control.

Master input 524 additionally comprises an engage control 540. Becauseof the size and force that many actors, particularly large industrialrobot arms carrying heavy cameras moving at up to several meters persecond, are capable of conveying in a collision, safety controls isextremely important for many embodiments of a motion controlled scene.Engage control 540 acts as an input regulator to prevent master input524 from being operated by accident, such that engage control must bedepressed at all times prior to any other input command being conveyedand acted upon from master input 524.

As shown in FIG. 5a , master input 524 also comprises a mode select 530and an analog interface 520. Analog interface 520 is any input devicecapable of defining a range of inputs by a user control. In onepotential embodiment, analog interface 520 is a wheel with a springaction that returns the wheel to a central position in the absence ofuser manipulation. In other potential embodiments, analog interface is alever, a sliding tab, or any other input control to enable a user toinput a signal. Master input 524 may comprise multiple input modes. Inone potential embodiment, master input 524 comprise a reset mode, a playmode, and a scan mode, with the mode selectable via mode select 530.

In a reset mode, operation of engage control 540 and analog interface520 operates to cause each actor within a scene to move to an initialposition for the beginning of a global timeline. Additionally, aspecific scene or frame number is selected by use of display interface510, and operation causes each actor to move to a position associatedwith that frame or time in the global timeline. Such a mode may allowdevice actors that are out of position for a particular time to be resetto a correct position.

In a play mode, operation of analog interface 520 may serve to speed upor slow down progress through a scene in a global timeline. For example,in a scene with actors set in position at global time a, selectingengage control 540 may serve to begin the action of all actors throughthe global timeline at a base rate, where each second of time isassociated with one second of progress through the global timeline.Operation of analog interface 520 in a first direction then serves toslow down progress through the global timeline, such that 1 second ofreal time is associated with 0.5 seconds of progress through the globaltimeline. If the analog interface 520 is then set back to center, theprogress through the global timeline will revert to a one to one rationwith real time, but with the remaining actions being delayed from theinitial start by the previous slowdown. Conversely, if analog interface520 is operated in a second direction opposite from the first direction,progress through the global timeline is increased. If, for example, thenominal time from time a to time b is 10 seconds, increasing progressthrough the global timeline by 10% may reduce that actual time requiredfor the motion controlled scene to progress from the positions of time ato the positions of time b by approximately 0.9 seconds and the actualtime required the progress from time a to time b with analog interfaceset to increase playback being approximately 9.1 seconds. This mayprovide use when a human player being recorded by a camera as part of amotion controlled scene is delivering lines more slowly or more quicklythan expected, and there is a desire to match the actions of the motioncontrolled scenes with the actions of human players that are not motioncontrolled.

In a scan mode, selecting the engage control 540 and then operatinganalog interface 520 may operate to shuttle or scan forwards orbackwards through the global timeline in a continuous fashion. Forexample, if a motion controlled set currently has actors in positionsassociated with time c, selecting shuttle mode and operating analoginterface 520 in a first direction may cause all actors to movecontinuously forward through the positions associated with the globaltimeline to reach time d. Moving analog interface 520 in a seconddirection may cause all actors to move continuously backwards throughthe positions associated with the global timeline to reach the positionsassociated with time b. In one potential embodiment, a particular timeor frame is selected using display interface 510. Operation of analoginterface 520 may shuttle continuously through the positions of theglobal timeline until the particular time or frame is reached. Masterinput 524 may then cease to control device actors until a selection indisplay interface 510 is activated acknowledging that the previouslyselected point has been reached, or until analog interface 520 isreturned to a central position.

FIG. 6a describes a block diagram of a motion control system 600. Motioncontrol system 600 comprises a master control 610, as well as details ofone potential embodiment of input, feedback, and device actorsub-systems. In the embodiment disclosed by motion control system 600,master control 610 comprises an operating system 614, master controllogic 612, master clock 616, network support 618, control logic 696, andfeedback 698. The elements of master control 610 are implemented in acomputer system comprising a general function processor and memoryhardware system. Alternatively, master control 610 is implemented incustom designed processing, memory, and networking hardware devices, oris implemented in an abstracted software layer of a general purposecomputing system. Master clock 616 functions as a real time clock tocoordinate movement of the actors in the system. Master control logic612 functions to integrate the individual control signals into a mastertimeline, and to correctly route control signals to the correct device,both during operation of the entire motion control scene through themaster timeline, and through individual modification and setting ofdevice actor positioning and function using control logic 696. Mastercontrol logic 612 also assists in coordination and routing of feedback698. Feedback 698 may include actual position and setting signalsreceived from monitors attached to device actors. One potential exampleis a location device attached to a robot arm. The actual position of thearm is tracked via the location device to provide feedback andcalibration in relation to an input position command sent to the armfrom either database 622, software control 657, or another control inputfor the robot arm. Operating system 614 includes special libraries andfunctionality for interacting with device actors, and may further serveto manage basic hardware computing functionality such as memory storageand processor usage. Operating system 614 may further enable networkingcapability via OS network 654 to communicate with various relateddevices.

Network support 618 may also enable communications from master control610 to related devices, actors, and input controls via network 620. Inone potential embodiment, network 620 may comprise an EtherCAT networkoperating according to IEEE 1588. In such an embodiment, packets are nolonger received, then interpreted and copied as process data at everyconnection. Instead, the frame is processed on the fly using a field busmemory management unit in each slave node. Each network node reads thedata addressed to it, while the telegram is forwarded to the nextdevice. Similarly, input data is inserted while the telegram passesthrough. The telegrams are only delayed by a few nanoseconds. On themaster side commercially available standard network interface cards oron board Ethernet controller can be as hardware interface. The commonfeature of these interfaces is data transfer to the master control viadirect memory access with no CPU capacity is taken up for the networkaccess. The EtherCAT protocol uses an officially assigned Ether Typeinside the Ethernet Frame. The use of this Ether Type allows transportof control data directly within the Ethernet frame without redefiningthe standard Ethernet frame. The frame may consist of severalsub-telegrams, each serving a particular memory area of the logicalprocess images that can be up to 4 gigabytes in size. Addressing of theEthernet terminals can be in any order because the data sequence isindependent of the physical order. Broadcast, Multicast andcommunication between slaves are possible. Transfer directly in theEthernet frame is used in cases where EtherCAT components are operatedin the same subnet as the master controller and where the controlsoftware has direct access to the Ethernet controller. Wiringflexibility in EtherCAT is further maximized through the choice ofdifferent cables. Flexible and inexpensive standard Ethernet patchcables transfer the signals optionally in Ethernet mode (100BASE-TX) orin E-Bus (LVDS) signal representation. Plastic optical fiber (POF) canbe used in special applications for longer distances. The completebandwidth of the Ethernet network, such as different fiber optics andcopper cables, can be used in combination with switches or mediaconverters. Fast Ethernet (100BASE-FX) or E-Bus can be selected based ondistance requirements. The Fast Ethernet physics enables a cable lengthof 100 m between devices while the E-Bus line is intended for modulardevices. The size of the network is almost unlimited since up to 65535devices can be connected.

Further, such an embodiment using EtherCAT supports an approach forsynchronization with accurate alignment of distributed clocks, asdescribed in the IEEE 1588 standard. In contrast to fully synchronouscommunication, where synchronization quality suffers immediately in theevent of a communication fault, distributed aligned clocks have a highdegree of tolerance from possible fault-related delays within thecommunication system.

Thus, data exchange is completely hardware based on “mother” and“daughter” clocks. Each clock can simply and accurately determine theother clocks run-time offset because the communication utilizes alogical and full-duplex Ethernet physical ring structure. Thedistributed clocks are adjusted based on this value, which means that avery precise network-wide time base with a jitter of significantly lessthan 1 microsecond is available.

However, high-resolution distributed clocks are not only used forsynchronization, but can also provide accurate information about thelocal timing of the data acquisition. For example, controls frequentlycalculate velocities from sequentially measured positions. Particularlywith very short sampling times, even a small temporal jitter in thedisplacement measurement leads to large step changes in velocity. In anembodiment comprising EtherCAT, the EtherCAT expanded data types(timestamp data type, oversampling data type) are introduced. The localtime is linked to the measured value with a resolution of up to 10 ns,which is made possible by the large bandwidth offered by Ethernet. Theaccuracy of a velocity calculation then no longer depends on the jitterof the communication system. It is orders of magnitude better than thatof measuring techniques based on jitter-free communication.

Finally, in an embodiment where network 620 comprises EtherCAT, a hotconnect function enables parts of the network to be linked and decoupledor reconfigured “on the fly”. Many applications require a change in I/Oconfiguration during operation. The protocol structure of the EtherCATsystem takes account of these changing configurations.

As described in FIG. 6a , network 620 then interfaces with individualdevices and actors via control data interface 650. Control datainterface is part of network 620, or may comprise distributed componentsat the input of individual actors. Additionally, Input and feedback datainterface is the same device as control data interface, acting as anopposite direction data flow of the same device interfacing with network620. The actors interfacing with network 620 comprise camera control632, secondary control 634, audio 636, digital output bank 648, camerarobot 670, follow focus 672, light 674, master input 650, and mastersafety control 690.

In certain embodiments, some actors may communicate directly to network620 via control data interface 650. For example, in the embodimentdescribed above where network 620 is an EtherCAT network, camera control632, secondary control 634, audio 636, and digital output bank 648 isable to communicate with no adapter through network 620. In such anembodiment, adapter 662 a is an EtherCAT-provibus adapter forcommunicating with camera robot 670, adapter 662 b is anEtherCAT-preston adapter for communicating with follow focus 672, andadapter 662 c is an EtherCAT-dmx adapter for controlling light 674.

In addition to master control 610 and the devices communicating withmaster control 610 via network 620, motion control system 600 maycomprise a plurality of other input, feedback, and devices such as OSnetwork 654, database 622, database interface 652, MI display 656,software control 657, video output 658, and adapters 655 a-c. In onepotential embodiment, OS network 654 is an Ethernet network coupled tomaster control 610. MI display 656 may function as a display for masterinput 650 in a manner similar to the display described for master input524 of FIG. 5a . Software control 657 may function as a softwareenvironment that may virtually create a global timeline for use withmaster control 610, and may also function to create a global timeline inreal-time with live control of device actors. In one embodiment,software control 657 is a software program such as MAYA™ with supportingsoftware add-ons. Finally, video output 658 is a real time video outputassociated with a camera in motion control system 600. As motion controlsystem moves through a global timeline, video output may allow real timevideo review of a motion controlled set, such that individual deviceactors is modified, or a master input 650 controls is adjusted based oninformation from video output 658.

Motion control system 600 of FIG. 6a may then allow for function andcreation of media in a variety of different ways, and described by FIG.6b . In an embodiment described by FIG. 6b , a user may create controldata using a software interface such as software control 657. Thecontrol data may then be stored in database 622 for future use.Alternatively or concurrently, the control data is sent to mastercontrol 610. The master control then manages control logic 696 andcreates a global timeline, such that any data required by specificdevices prior to function is communicated to those devices. For example,in one potential embodiment, camera robot 670 must have all positiondata prior to operation, with camera robot 670 functioning insynchronization with the global timeline by receiving a time or frameinput and acting in response using previously received controlinformation to move to the correct position for the current state of theglobal timeline. After the global timeline is prepared by master control610, and all device actors are prepared, a user may use a master input650 or a software control 657 to send an operation signal to mastercontrol 610. The master control then sends the appropriate controlsignals to individual device actors given the global timeline positionand synchronization. The device actors then operate according to controlsignals, and send feedback signals via network 620 documenting actualoperation, errors, and/or safety shutdown conditions. The feedback datacan then be used to update the control signal managed by master control610 to modify or match the original control data from the database 622or software control 657.

In one potential embodiment, motion control system 600 comprises mastersafety control 690. Master safety control may comprise a hard wired shutdown control connected directly to device actors identified as posing asafety risk. Such a shutdown control is attached to master input 650. Inanother potential embodiment, master safety control 690 comprises asafety computing system or a safety PLC attached to safety sensors. Suchsafety sensors may comprise object proximity detectors attached to adevice actor. When a scene is created in motion control system 600, thescene will have data for expected device actors and for playersappearing in the scene. Object proximity detectors is programmed orcoupled to a computing system to filter expected objects with expectedlocations from unexpected objects and locations. If an object isdetected by an object proximity detector in an unexpected and unsafelocation, a signal is sent to shut down the entire motion controlledscene, or at a minimum, device actors determined to pose a potentialdanger to the unexpected object. In another potential embodiment,boundaries for a motion controlled scene are determined. Proximitydetectors or motion detectors is configured to detect movement across ascene boundary or a device actor boundary during operation of the motioncontrol system, and to halt operation when an unsafe motion or object isdetected. In this fashion, master safety control may observes an areaproximate to various device actors and transmit a safety shutdownsignal. In one potential embodiment, an object detector comprises alight detection and ranging unit (LIDAR). In another potentialembodiment, the object detector is a passive infrared sensor (PIR).

FIG. 7 shows a block diagram of an exemplary computer apparatus that canbe used in some embodiments of the invention (e.g., in the componentsshown in the prior Figures).

The subsystems shown in FIG. 7 are interconnected via a system bus 710.Additional subsystems such as a printer 708, keyboard 718, fixed disk720 (or other memory comprising computer readable media), monitor 714,which is coupled to display adapter 712, and others are shown.Peripherals and input/output (I/O) devices, which couple to I/Ocontroller 702, can be connected to the computer system by any number ofmeans known in the art, such as through serial port 716. For example,serial port 716 or external interface 722 can be used to connect thecomputer apparatus to a wide area network such as the Internet, a mouseinput device, or a scanner. The interconnection via system bus 710allows the central processor 706 to communicate with each subsystem andto control the execution of instructions from system memory 704 or thefixed disk 720, as well as the exchange of information betweensubsystems. The system memory 704 and/or the fixed disk 720 may embody acomputer readable medium.

FIG. 8 illustrates one potential embodiment of a user interface 1200 ina software control for use with a motion control photography system.Such an interface may enable a number of functions, including modelingof individual device actors, software control of individual deviceactors, creation of control data for individual device actors, andrecording control or operation data for individual device actors.Similarly multiple devices is modeled simultaneously to model, control,or record operation of an entire motion controlled set using aninterface such as user interface 1200.

User interface 1200 may include an actor panel with a plurality ofinterface buttons including an “actors” drop down menu to show a list ofthe current device actors is a scene, and “add actor” scene to bring upa prompt which allows selections of a name and/or type of device actorto be created or added to a scene, and a “delete actor” interface toallow deletion of an actor from a scene.

User interface 1200 may also include a selection panel. A selectionpanel may include an interface for “Select End Effector” which selects acontroller on the robot which allows the user to drive the robot via awrist. This controller, when selected and viewed through channel boxeditor, may house additional attributes that the user can control. Theseattributes may include:

-   -   Visibility—Toggles the visibility of the controller in a view        port.    -   Disable Heads up Display (HUD) Updates—Disables all HUD updates        in a view port.    -   A(1-6) Angle—Displays the current angle of each respective robot        axis.    -   Track Robot (TR) Position—Displays the position of robot on the        track axis.    -   Show Axes—Toggles the visibility of a skeleton in a view port.    -   FK IK (A1-A6)—Allows the switching settings for of individual        joints.

A selection panel may also include a “select base control” interfacewhich may select a controller on the robot which allows the user toposition the robot from the base. Such a controller may, when selectedand viewed through the channel box editor, houses additional attributesthat the user can manipulate such as:

-   -   Translate Z—Moves the base along the track system.    -   Rotate Y—Changes the orientation of the base.    -   Column Height—Changes the base height of the robot.

A selection panel may also include additional interface controls suchas:

-   -   Select Track Control—Selects a controller on the robot which        allows the user to move and orient the track system to their        liking. further describe the disconnect between the end effector        and this node.    -   Select Worldspace Mount—Selects a controller for adding a mount.    -   Select World Controls—Selects a master controller which allows        the user to move and orient the entire robot and track system        together.

Additionally, user interface 1200 may provide controls for adding andremoving mounts to device actors within the software system, and forsetting placement for mounts that are added to device actors. Forexample, FIG. 8 shows three potential positions for mounting a camera toa robot arm that is a device actor in the current system. The userinterface 1200 may provide graphical or number input selections forchoosing between various mounting options, and for adjusting theposition variables associated with such a setting.

In one potential embodiment of a motion control photography system, userinterface 1200 is integrated through a software control to a camerarobot such as software control 657 and camera robot 670 of FIG. 6a toenable an automated system calibration. One potential such calibrationcomprises a lens node calculation for determining a calibrated lensposition in a camera mounted to a robot arm. One potential embodiment ofsuch a calibration comprises providing software control 657 withinformation regarding the location and dimensions of a base, wrist orjoint elements, and camera mounts for camera robot 670. A calibrationplayer with known characteristics such as location and size is alsoprovided. When a calibrate command is selected via software control 657,camera robot 670 records a plurality of images of the calibration playerfrom different camera angles or locations. The images in conjunctionwith known data regarding the calibration player and camera robot 670allows the lense node of the currently mounted camera to be calculatedand incorporated into the device actor modeled in software control 657.

In an alternative embodiment of a calibration, software interface 1200may have a calibration command for altering size or location data inresponse to feedback from a temperature sensor. For example, softwarecontrol 657 is receive data via digital output bank 658 from atemperature sensor attached to a rail that is part of camera robot 670.Software control 657 is configured to adjust command signals due toknown changes in rail characteristics over temperature, or is configuredto signal a need to measure changes in device actor locations inresponse to temperature changes.

A software control such as software control 26 of FIG. 1 or softwarecontrol 657 of FIG. 6a may also comprise an analysis interface such asanalysis interface 1300 of FIG. 9. Analysis interface 1300 showsmovement data for individual axis of a device actor. Data such ascurrent velocity and current acceleration is displayed, along withgraphs of these characteristics over time through a scene, maximumallowable limits for each characteristic, and maximum achieved value foreach characteristic. The values presented in analysis interface 1300 aremodeled from a global timeline created within a software control orrecorded in a database. The values may also be values recorded from theactual physical motion of device actors playing through a scene. Use ofsuch analysis data may provide for modifications to be made to a scenewhile quickly verifying that safe limits of device actor function arenot mistakenly exceeded.

In addition to analysis of operation of individual device actors,software control is used for preventative safety in detecting potentialcollisions between device actors in modeling the motion of the actorsthrough a global timeline. Further, such modeling of a scene through aglobal timeline is used to set safety parameters for a safety systemsuch as master safety control 690 of FIG. 6a . Modeling of locations andvelocities of device actors through a global timeline may enableidentification of unsafe zones and unsafe times in an area or set arounda motion control system. Such an identification is used to set sensingtriggers of object detectors that are part of the safety systemdescribed above. For example, if an area within 5 feet of a certaindevice actor is determined to be at risk of collision, and a buffer zoneof 10 additional feet is required to insure safety during operation, aLIDAR detector is configured to detect unexpected objects and movementwithin a 15 foot area of the device actor during operation, and toautomatically create a safety shutdown if an object is detected. In analternative embodiment, the LIDAR detector is configured to create awarning signal if an object is detected in a periphery of the dangerzone, and only to create a shutdown if the detected object is movingtoward a potential impact zone.

In an alternate embodiment, a software control includes modeling ofactors and models of defined safe zones. Analysis of the motion of theactors in software control allows a modeled safety check to see if anyactor collides with a defined safe zone. Safe zones are defined by entryof fixed volumes of space into software control, by image capture of aset location. Safe zones may also be defined to be variable based on adetected motion, jerk, velocity, or acceleration of an object in a safezone. In an alternate embodiment a safe zone is defined by input fromtransponder device data. For example, a transponder location device isattached to a player, and a safe zone defined by a distance from thetransponder. The transponder feeds location data to software control,which may update safe zones within a software control or within a mastersafety control. In another embodiment, fixed safe zones are definedwithin software control, and published prior to a safety PLC withinmaster safety control prior to operation of the motion controlled set.

FIG. 10 describes an alternate embodiment of a motion control systemaccording the present innovations. The function of and operation ofmotion control system 700 is understood in conjunction with the detaileddescription of motion control systems described above.

The previous description of the embodiments is provided to enable anyperson skilled in the art to practice the invention. The variousmodifications to these embodiments will be readily apparent to thoseskilled in the art, and the generic principles defined herein areapplied to other embodiments without the use of inventive faculty. Thus,the present invention is not intended to be limited to the embodimentsshown herein, but is to be accorded the widest scope consistent with theprinciples and novel features disclosed herein.

Embodiments of the invention are not limited to the above-describedembodiments. For example, throughout the description, various examplesof device actors and robots are presented for use in a motion controlphotography system The embodiments are not limited to the general orspecific examples provided herein, but may include other embodimentsapparent to a person of ordinary skill in the art from the abovedescriptions. By further example, the above description provides anillustration of three modes selectable as input modes for a masterinput. The embodiments of the invention are not limited to these modes,but additional modes is used that may differ from the modes presented.

It should be understood that the present invention as described abovecan be implemented in the form of control logic using computer softwarein a modular or integrated manner. Based on the disclosure and teachingsprovided herein, a person of ordinary skill in the art can know andappreciate other ways and/or methods to implement the present inventionusing hardware and a combination of hardware and software.

Any of the software components or functions described in thisapplication, is implemented as software code to be executed by aprocessor using any suitable computer language such as, for example,Java, C, or Perl using, for example, conventional or object-orientedtechniques. The software code is stored as a series of instructions, orcommands on a computer readable medium, such as a random access memory(RAM), a read only memory (ROM), a magnetic medium such as a hard-driveor a floppy disk, or an optical medium such as a CDROM. Any suchcomputer readable medium may reside on or within a single computationalapparatus, and is present on or within different computationalapparatuses within a system or network.

The above description is illustrative and is not restrictive. Manyvariations of the invention will become apparent to those skilled in theart upon review of the disclosure. The scope of the invention should,therefore, be determined not solely with reference to the abovedescription, but instead should be determined with reference to thepending claims along with their full scope or equivalents.

One or more features from any embodiment is combined with one or morefeatures of any other embodiment without departing from the scope of theinvention. For the purposes of the present application, a recitation of“a”, “an” or “the” is intended to mean “one or more” unless specificallyindicated to the contrary. Descriptions of alternative embodimentsrecite that certain portions of an embodiment “may” be present in thespecific alternative embodiment being described, and that certainembodiments include the described element.

In addition to the above described embodiments, certain implementationsinvolve motion control combined with 3D projection mapping. In 3Dprojection mapping, a viewer is presented with the visual projection anenvironment that is not actually physically present, when light isprojected on a surface that is rendered from a specific perspective.

In 3D projection mapping, the knowledge of the physical spatialdimensions and coordinates of the projection surface or object isnecessary to achieve the desired effect. Traditionally, thesedimensions/location are stationary, and are therefore not varying.

In one potential embodiment according to the present innovations, theprojection surface is in motion through a volume. Specifically thesurface is moved via robotic (servo) actuation whereby the exactphysical location of the projection surface is known to the system. Inaddition to the concept of moving the projection surface via acontrolled method, embodiments also include movement of the projectorvia a controlled method to known coordinates. This might serve forexample to provide an accurate representation from a correctly renderedperspective to a viewer that is on a motion platform (i.e., a passengercar on a ride or exhibit).

By placing a projection surface on the end effector of a roboticplatform or arm, certain embodiments of the system are able to move theprojection surface or object to a specific known coordinate in x,y,z(Cartesian) space. Since the exact location of the object is known(through robot encoder feedback, as well as programmed position), we areable to project imagery on the surface through mapping in 3D (virtual)space (in real-time).

Embodiments of the system thus create this effect in a traditional 3Dprojection mapping methodology whereby we create a 3D model (virtual)representation of the projection surface/object to calculate the properrender considering point of view and perspective. This render is thenprojected as light onto the object.

Such a system is incorporated with any of the above described softwarecontrol or master control systems to create an audiovisual presentationthat is controlled, programmed, and modified to enable improved editingand creation of 3D projection mapped audiovisual presentations that areintegrated with lighting, safety, and software systems described above.

FIG. 11 describes one potential embodiment of a 3D projection mappedaudiovisual presentation system. FIG. 11 shows device actor 1142,projector 1150 mounted on device actor 1142, device actor 1144, andprojection surface 1140 mounted on device actor 1144. Projector 1150 mayreceive or include signal information for projecting an image ontoprojection surface 1140 as projection surface 1140 is moved through aset of spatial coordinates by device actor 1144. In alternativeembodiments, both device actors 1142 and 1144 is moving through spatialcoordinates, or only device actor 1142 is moving. The spatialcoordinates is provided by a master control system, a computer, or anyother known means for controlling a device actor. As part of the system,the image projected onto the surface 1140 is created based on the knowninformation of the movement of the device actor 1144 through the set ofspatial coordinates.

For example, device actors 1142 and 1144 is coupled to a softwarecontrol system such as software control 26 of FIG. 1 or software control726 with video output 702 of motion control system 700. The system hasimbedded information related to the projection surface 1140 and adesired visual representation. A manual control allows a user tomanually adjust projection surface 1140's spatial position, and asoftware control is programmed to automatically adjust a graphicaloutput from the projector to create the desired visual presentation.Alternately, a software control uses animation to describe a desiredvisual representation, and the software control automatically adjustsprojector 1150 and projection surface 1140 using device actors 1142 and1144 to create the desired audiovisual representation on projectionsurface 1140.

In one potential embodiment, and animation control system coupled to aprojector projects an animation onto a projection surface. The animationcontrol system includes an interface for modifying a 3D animation of theprojection. The animation control system may include a relativecoordinate representation of the projection surface and the currentlocation of the projection surface based on control information receivedfrom a device actor that physically controls the location of theprojection surface. The interface may thus enable modification of theanimation within the relative coordinate system presented for theprojection surface. As the projection surface is moved by a deviceactor, the animation control system receives a signal indicating thismovement. The system then compensates based on any adjustment angle tocreate the appropriate presentation and perspective of the 3Dprojection. For example, as the projection surface angles away from aposition normal to the projector, the projection is transformed topresent a narrower output from the projector that will spread across theangled projector to maintain a constant image from the projectionsurface. The optical output power from the projector may similarly beincreased to maintain a constant brightness from the projection surface.The output signal from the animation control system will automaticallyadjust the projection at the projector to maintain the animation at theappropriate location in the relative coordinate system on the projectionsurface. In one embodiment, multiple projectors may project the same 3Danimation onto the projection surface at the same or alternating times.If the projection surface rotates such that a first projectors opticalpath to the projection surface is blocked, a second projector in asecond location with an unblocked optical path may then project the same3D animation onto the projection surface.

In additional embodiments, set models and movement of a set through ascene may further include modeling of an optical path from a projectorto a projection surface. A master control or animation computer systemthat adjusts a set and scene movements may create an alert if changes tothe set and scene create an obstruction of an optical path that wouldblock part of the 3D projection and prevent presentation of the 3Dprojection at the projection surface.

FIGS. 12-21 show additional details of potential implementations of a 3Dmotion control projection system in accordance with the presentinnovations. FIG. 12 shows one potential implementation of a 3Dprojection with projector 1150 projecting onto projection surface 1140.FIGS. 13-19 show one implementation of a projection surface 1140 mountedon a device actor 1144. In these figures, a roughly spherical projectionobject has a computer generated face projected onto the surface. This isan example of a projection. As the device actor moves the projectionsurface, the projected image of the face maintains an appropriateposition on the projection surface, and the features of the face areanimated to produce blinking eyes, a moving mouth, and other 3Danimation on the moving projection surface. This may be matched to a setof coordinates associated with the projection surface. The projectionsurface 1140 is moved or translated through a set of coordinates, andthe relative location of the face projection on projection surface 1140may remain stable within a projection coordinate system on theprojection surface as the projection surface is moved through the set ofcoordinates in space. This essentially presents the illusion ofanimation and mobility with the potential for extremely complex facialanimation beyond what is possible using mechanical facial animation.FIGS. 20 and 21 show a computer model of the projection surface in auser interface 2010 in a display. The projection that operatesconcurrently with the motion of the device actor and the projection ofthe image onto the projection screen. The software may enablevisualization and use of spatial information from the device actor, andadjustment or control of the projected image in real time or near realtime.

In various embodiments, the projection surface is then be integratedwith a scene as described above in the master control system. Movementof the entire scene is be matched to the timeline, and the animation ofthe 3D projection is additionally matched to the timeline, such thatwhen manual or control system adjustments to the speed of the scene aremade, the animation and movement of the 3D projection surface are alsoadjusted and matched to the movement of the entire scene. In onepotential embodiment, a projection surface is a non-planar projectionsurface. In an alternate embodiment, the projection surface is a rearprojection surface. In another alternate embodiment, a set includes aplanar projection surface and a non-planar 3D projection surface.

In additional embodiments, the 3D computer animated projection is bothpart of a scene in a master control timeline, but also have certainportions of the animation matched to a real time input. For example, ifa voice and spoken words from an actor are presented as coming from theanimated device actor, the 3D projection of mouth movement is matched inreal time to the words spoken by the human actor. Movement of theprojection surface and certain parts of the 3D projection animation suchas eye blinking is predetermined and automated, while projection of themouth movement is specifically adapted to a human actor that is hiddenfrom the audience.

In various embodiments, then a system for 3D projection with motioncontrol comprises a projector; and a projection surface coupled to afirst device actor, wherein the first device actor moves the projectionsurface through a set of spatial coordinates, and a 3D projection fromthe projector is projected onto a set of coordinates of the projectionsurface and matches the 3D projection to the set of coordinates of theprojection surface as the projection surface moves through the set ofspatial coordinates. Additional alternative embodiments comprise ananimation computer system coupled to the projector that maps the 3Dprojection onto the projection surface and animates the 3D projectionwhile matching the 3D projection to the set of coordinates of theprojection surface as the non-planar projection surface moves throughthe set of spatial coordinates. In alternative embodiments, animationcomputer system is further coupled to the device actor, and theanimation computer system provides the set of spatial coordinates to thedevice actor to instruct the device actor to move the non-planarprojection surface through the set of spatial coordinates.

Additional alternative embodiments comprise a master control thatreceives a plurality of control signals comprising control data for aplurality of device actors including the first device actor andsynchronizes the plurality of control signals with a global timeline tocreate a plurality of synchronized signals, such that control data foreach actor of the device actors is associated with a correspondingposition in the global timeline; and a master input that conveys amaster input signal to the master control indicating a position in theglobal timeline and a rate of progression through the global timeline;wherein the master control responds to the master input signal bycommunicating the plurality of synchronized signals associated with theposition in the global timeline to the plurality of device actors, andthe control data for each actor of the device actors is sent torespective device actors at the rate of progression through the globaltimeline.

In additional alternative embodiments, the master input comprises ananalog input, a mode selector, and an analog input action control; theanalog input action control returns the analog input to a first positionwhen no force is applied to the analog input; for a first selected modeof the master input, adjustment of the analog input in a first directionincreases the rate of progress through the global timeline from apredetermined standard forward rate; for the first selected mode of themaster input, adjustment of the analog input in a second directiondifferent from the first direction decreases the rate of progressthrough the global timeline from the predetermined standard forwardrate; for a second selected mode of the master input, adjustments to theanalog input in the first direction creates a variable rate of progressin the forward direction from a stationary rate; and for the secondselected mode of the master input, adjustments to the analog input inthe second direction creates a variable rate of progress in a reversedirection from the stationary rate.

In additional alternative embodiments, the animation computer systemcomprises a real time input for modifying the 3D projection whilemaintaining the 3D projection within the set of coordinates of theprojection surface as the projection surface moves through the set ofspatial coordinates.

Additional alternative embodiments comprise an object detector thatobserves an area proximate to a first device actor of the plurality ofdevice actors and transmits a safety shutdown signal to the first deviceactor when the object detector senses an object.

Additional alternative embodiments comprise a master safety control;wherein the master safety control comprises a programmable logiccircuit; and wherein the safety shutdown signal is transmitted to eachof the plurality of device actors via the programmable logic circuit.

In additional alternative embodiments, the first device actor is arobotic arm, the object detector is attached to a mounting point of therobotic arm, and wherein the area proximate to the first device actorcomprises an area extending a fixed distance from the object detector.

In additional alternative embodiments, the area proximate to the firstdevice actor varies over time in a predetermined manner determined inassociation with the position in the global timeline and the rate ofprogress through the global timeline.

Another alternative embodiment comprises a method for motion control.One embodiment of the method comprises moving a projection surfacethrough a set of spatial coordinates using a first device actor coupledto the projection surface; projecting from a projection device using asignal from an animation computer, a 3D projection onto the projectionsurface; and mapping, using the animation computer, the 3D projectiononto the projection surface to match a 3D projection movement with themovement of the projection surface through the set of spatialcoordinates.

Additional alternative embodiments comprise receiving control data for aplurality of device actors including the first device actor at a mastercontrol; synchronizing a plurality of control signals, using the mastercontrol, with a global timeline to create a plurality of synchronizedsignals, such that the control data for each actor of the device actorsis associated with a corresponding position in the global timeline;receiving, at the master control, a master input signal indicating aposition in the global timeline and a rate of progression through theglobal timeline; and communicating, in response to the master inputsignal, the plurality of synchronized signals associated with theposition in the global timeline from the master control to the pluralityof device actors, wherein the plurality of synchronized signals are sentto the plurality of device actors at the rate of progression through theglobal timeline indicated by the master input signal.

Additional alternative embodiments comprise detecting, using at leastone object detector, an area proximate to the plurality of deviceactors; and transmitting, from the at least one object detector, asafety shutdown signal that prevents the plurality of device actors fromoperating.

Additional alternative embodiments comprise methods where the rate ofprogress through the global timeline comprises a reverse rate ofprogress through the global timeline.

Additional alternative embodiments comprise receiving at the mastercontrol, during the communicating of the plurality of synchronizedsignals, a modifying control input for a first device actor of theplurality of device actors from an independent manual control;synchronizing the modifying control input with the plurality ofsynchronized signals to create an updated plurality of synchronizedsignals during the communication of the plurality of synchronizedsignals; and communicating the updated plurality of synchronized signalsin place of the synchronized signals.

Additional alternative embodiments comprise storing the updatedplurality of synchronized signals in a database, wherein the controldata is received at the master control from the database.

Additional alternative embodiments comprise receiving at the mastercontrol, feedback data from the plurality of device actors describing anoperating characteristic associated with the synchronized controlsignals; and modifying the synchronized control signals in response tothe feedback data.

Additional alternative embodiments comprise adjusting in real time, thesignal from the animation computer, to create a real time computeranimation at the projection surface based on a user input at theanimation computer.

Additional alternative embodiments comprise a computer readable mediumstoring instructions for execution by a computing system forimplementing a method of creating a film using motion control, themethod comprising: modeling physical characteristics and locations,using a software control comprising a computing device; for a pluralityof device actors, at least one projector, and at least one projectionsurface in a motion control set; modeling movement characteristics andan optical path from the at least one projector to the at least oneprojection surface, using the software control, for the plurality ofdevice actors in the motion control set to create control data for theplurality of device actors; analyzing the modeling of the movements,physical characteristics, and locations of the plurality of deviceactors, using the software control, to detect collisions, obstruction ofthe optical path, and to detect device actor motion that exceeds a setof predetermined operating limits; and communicating the control data toa master control that synchronizes the control data and transmits thecontrol data to the plurality of device actors.

In additional alternative embodiments, the method further comprisesanalyzing the modeling of the movements, physical characteristics, andlocations of the plurality of device actors, using the software control,to determine a set of locations proximate to the device actors thatinclude a collision risk and transmitting the set of locations to amaster safety control comprising at least one object detector.

What is claimed is:
 1. A system for 3D projection with motion controlcomprising: a projection surface coupled to a first device actor,wherein the first device actor moves the projection surface through aset of spatial coordinates; a projector that is operable to project a 3Dprojection onto a set of coordinates of the projection surface; and ananimation computer system that matches the 3D projection to the set ofcoordinates of the projection surface as the projection surface movesthrough the set of spatial coordinates, such that movement of the 3Dprojection matches movement of the projection surface during at leasttranslational movement of the projection surface from a first locationin the set of spatial coordinates to a second location in the set ofspatial coordinates.
 2. The system of claim 1, wherein the animationcomputer system is coupled to the projector that maps the 3D projectiononto the projection surface and animates the 3D projection whilematching the 3D projection to the set of coordinates of the projectionsurface as the non-planar projection surface moves through the set ofspatial coordinates.
 3. The system of claim 2 wherein the animationcomputer system is further coupled to the device actor, and theanimation computer system provides the set of spatial coordinates to thedevice actor to instruct the device actor to move the non-planarprojection surface through the set of spatial coordinates.
 4. The systemof claim 3 further comprising: a master control that receives aplurality of control signals comprising control data for a plurality ofdevice actors including the first device actor and synchronizes theplurality of control signals with a global timeline to create aplurality of synchronized signals, such that control data for each actorof the device actors is associated with a corresponding position in theglobal timeline; and a master input that conveys a master input signalto the master control indicating a position in the global timeline and arate of progression through the global timeline; wherein the mastercontrol responds to the master input signal by communicating theplurality of synchronized signals associated with the position in theglobal timeline to the plurality of device actors, and the control datafor each actor of the device actors is sent to respective device actorsat the rate of progression through the global timeline.
 5. The system ofclaim 4 wherein the master input comprises an analog input, a modeselector, and an analog input action control; wherein the analog inputaction control returns the analog input to a first position when noforce is applied to the analog input; wherein for a first selected modeof the master input, adjustment of the analog input in a first directionincreases the rate of progress through the global timeline from apredetermined standard forward rate; wherein for the first selected modeof the master input, adjustment of the analog input in a seconddirection different from the first direction decreases the rate ofprogress through the global timeline from the predetermined standardforward rate; wherein for a second selected mode of the master input,adjustments to the analog input in the first direction creates avariable rate of progress in the forward direction from a stationaryrate; and wherein for the second selected mode of the master input,adjustments to the analog input in the second direction creates avariable rate of progress in a reverse direction from the stationaryrate.
 6. The system of claim 5 wherein the animation computer systemcomprises a real time input for modifying the 30 projection whilemaintaining the 30 projection within the set of coordinates of theprojection surface as the projection surface moves through the set ofspatial coordinates.
 7. The system of claim 4 further comprising anobject detector that observes an area proximate to a first device actorof the plurality of device actors and transmits a safety shutdown signalto the first device actor when the object detector senses an object. 8.The system of claim 7 further comprising a master safety control;wherein the master safety control comprises a programmable logiccircuit; and wherein the safety shutdown signal is transmitted to eachof the plurality of device actor via the programmable logic circuit. 9.The system of claim 7 wherein the first device actor is a robotic arm,the object detector is attached to a mounting point of the robotic arm,and wherein the area proximate to the first device actor comprises anarea extending a fixed distance from the object detector.
 10. The systemof claim 8 wherein the area proximate to the first device actor variesover time in a predetermined manner determined in association with theposition in the global timeline and the rate of progress through theglobal timeline.
 11. A method comprising: projecting from a projectiondevice, a 3D projection onto a set of coordinates of a projectionsurface, wherein the projection surface is initially located at a firstlocation in a set of spatial coordinates; moving the projection surfacethrough the set of spatial coordinates using a first device actorcoupled to the projection surface, wherein the movement through the setof spatial coordinates comprises a translation of the projection surfacefrom the first location in the set of spatial coordinates to a secondlocation in the set of spatial coordinates; and mapping, using theanimation computer, the 3D projection onto the projection surface tomatch movement of the 3D projection with the movement of the projectionsurface during the movement of the projection surface the set of spatialcoordinates from the first location to the second location.
 12. Themethod of claim 11 further comprising: receiving control data for aplurality of device actors including the first device actor at a mastercontrol; synchronizing a plurality of control signals, using the mastercontrol, with a global timeline to create a plurality of synchronizedsignals, such that the control data for each actor of the device actorsis associated with a corresponding position in the global timeline;receiving, at the master control, a master input signal indicating aposition in the global timeline and a rate of progression through theglobal timeline; and communicating, in response to the master inputsignal, the plurality of synchronized signals associated with theposition in the global timeline from the master control to the pluralityof device actors, wherein the plurality of synchronized signals are sentto the plurality of device actors at the rate of progression through theglobal timeline indicated by the master input signal.
 13. The method ofclaim 12 further comprising: detecting, using at least one objectdetector, an area proximate to the plurality of device actors; andtransmitting, from the at least one object detector, a safety shutdownsignal that prevents the plurality of device actors from operating. 14.The method of claim 12 wherein the rate of progress through the globaltimeline comprises a reverse rate of progress through the globaltimeline.
 15. The method of claim 12 further comprising: receiving atthe master control, during the communicating of the plurality ofsynchronized signals, a modifying control input for a first device actorof the plurality of device actors from an independent manual control;synchronizing the modifying control input with the plurality ofsynchronized signals to create an updated plurality of synchronizedsignals during the communication of the plurality of synchronizedsignals; and communicating the updated plurality of synchronized signalsin place of the synchronized signals.
 16. The method of claim 15 furthercomprising storing the updated plurality of synchronized signals in adatabase, wherein the control data is received at the master controlfrom the database.
 17. The method of claim 12 further comprising:receiving at the master control, feedback data from the plurality ofdevice actors describing an operating characteristic associated with thesynchronized control signals; and modifying the synchronized controlsignals in response to the feedback data.
 18. The method of claim 11further comprising: adjusting in real time, the signal from theanimation computer, to create a real time computer animation at theprojection surface based on a user input at the animation computer. 19.A non-transitory computer readable medium storing instructions forexecution by a computing system for implementing a method of creating afilm using motion control, the method comprising: modeling physicalcharacteristics and locations, using a software control comprising acomputing device; for a plurality of device actors, at least oneprojector, and at least one projection surface in a motion control set;modeling movement characteristics and an optical path from the at leastone projector to the at least one projection surface, using the softwarecontrol, for the plurality of device actors in the motion control set tocreate control data for the plurality of device actors; analyzing themodeling of the movements, physical characteristics, and locations ofthe plurality of device actors, using the software control, to detectcollisions, obstruction of the optical path, and to detect device actormotion that exceeds a set of predetermined operating limits; andcommunicating the control data to a master control that synchronizes thecontrol data and transmits the control data to the plurality of deviceactors.
 20. The non-transitory computer readable medium of claim 19,wherein the method further comprises: analyzing the modeling of themovements, physical characteristics, and locations of the plurality ofdevice actors, using the software control, to determine a set oflocations proximate to the device actors that include a collision risk;and transmitting the set of locations to a master safety controlcomprising at least one object detect.